Data load system with distributed data facility technology

ABSTRACT

A source data file can be loaded into a database by extracting data from the source data file at a computing device, validating the data from the source data file at the computing device, transforming the data from the source data file at the computing device, requesting the data to be transferred to a first database at a first message processor module, delivering the data to the first database, and responding that the data has been transferred to the first database at the first message processor module to acknowledge receipt of the data in the database at the computing device. The data may be delivered to one or more databases using, for example, a plurality of distributed data facility (DDF) threads.

CROSS-REFERENCE TO RELATED APPLICATION

This application is related to U.S. patent application Ser. No.14/262,014, filed Apr. 25, 2014, and entitled DATA LOAD PROCESS. Therelated application is hereby incorporated by reference in its entirety.

FIELD

Aspects of the disclosure generally relate to a data load process. Morespecifically, aspects of the disclosure provide example systems andexample tools that provide a data load process using native storedprocedures to concurrently load multiple data files into a table ordatabase at one time.

BACKGROUND

Information technology infrastructures may require several operatingenvironments, vendor resource deployment, authentication repositoriesand mechanisms, application servers, and databases for storing,indexing, and updating massive amounts of data constantly. In order tooperate a large entity's information technology and be able to store,index, and manage data received by the entity, these systems andprocesses can be configured to work together.

Databases load data received by the entity for indexing continuously inorder to keep up with the volume of data an entity receives and/ortracks on a daily basis. Frequently, the data stored by the entity mayneed to be updated. This may occur for a variety of reasons, includingmistakes in the loading, changing flags, changes in the information, orthe like. The process of database loading and updating takes time,central processing units (CPU) away from the infrastructure, loggingtime, and in some cases has redundancies and other issues associatedwith the process.

BRIEF SUMMARY

The following presents a simplified summary of various aspects describedherein. This summary is not an extensive overview, and is not intendedto identify key or critical elements or to delineate the scope of theclaims. The following summary merely presents some concepts in asimplified form as an introductory prelude to the more detaileddescription provided below.

In one example, a source data file can be loaded into a database byperforming one or more of: (1) extracting data from the source data fileat a computing device, (2) validating the data from the source data fileat the computing device, (3) transforming the data from the source datafile at the computing device, (4) requesting the data to be transferredto a database at a message processor module, (5) delivering the data tothe database, or (6) responding that the data has been transferred tothe database at the message processor module to acknowledge receipt ofthe data in the database at the computing device.

In another example, a method may comprise one or more mid-range serversreceiving one or more data files. The one or more mid-range servers mayextract data from the one or more data files. A plurality of distributeddata facility (DDF) threads may be generated between the one or moremid-range servers and a first database. The one or more mid-rangeservers may send a first portion of the extracted data to the firstdatabase using the generated plurality of DDF threads. On the otherhand, a second portion of the extracted data may be sent to a messagequeue accessible to a second database.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the present disclosure and theadvantages thereof may be acquired by referring to the followingdescription in consideration of the accompanying drawings, in which likereference numbers indicate like features, and wherein:

FIG. 1 illustrates one example of a network architecture and dataprocessing device that may be used to implement one or more illustrativeaspects discussed herein;

FIG. 2 illustrates a schematic of an example database loading system;

FIG. 3 depicts an example process for loading data; and

FIG. 4 depicts a schematic of an alternative example database loadingsystem.

DETAILED DESCRIPTION

In the following description of the various embodiments, reference ismade to the accompanying drawings, which form a part hereof, and inwhich is shown by way of examples various examples in which thedisclosure may be practiced. It is to be understood that otherembodiments may be utilized and structural and functional modificationsmay be made without departing from the scope of the present disclosure.The disclosure is capable of other embodiments and of being practiced orbeing carried out in various ways. Also, it is to be understood that thephraseology and terminology used herein are for the purpose ofdescription and should not be regarded as limiting. Rather, the phrasesand terms used herein are to be given their broadest interpretation andmeaning. For example, the use of “including” and “comprising” andvariations thereof is meant to encompass the items listed thereafter andequivalents thereof as well as additional items and equivalents thereof,and the use of the terms “mounted,” “connected,” “coupled,”“positioned,” “engaged” and similar terms, is meant to include bothdirect and indirect mounting, connecting, coupling, positioning andengaging.

As noted above, various aspects of the disclosure relate to a system andmethod for data loading. In one example, the system is configured toconduct a data load process using native stored procedures toconcurrently load multiple data files into a table at one time. Filescan be sent to a server where they will be extracted, validated, splitinto units of work (e.g. 2500 records per job), and transformed into theproper format (e.g. XML load files). These files can be configured asmessages on a message processor module and picked up using a listenermodule, for example. Data can be loaded into temporary tables prior to afinal load. Before discussing these aspects in greater detail, however,several examples of network architectures and data processing devicesthat may be used in implementing various aspects of the disclosure willfirst be discussed.

FIG. 1 illustrates one example of a network architecture and dataprocessing device that may be used to implement one or more illustrativeaspects. Various network nodes 103, 105, 107, and 109A-F may beinterconnected via a wide area network (WAN) 101, such as the Internet.Other networks may also or alternatively be used, including privateintranets, corporate networks, LANs, wireless networks, personalnetworks (PAN), and the like. Network 101 is for illustration purposesand may be replaced with fewer or additional computer networks. A localarea network (LAN) may have one or more of any known LAN topology andmay use one or more of a variety of different protocols, such asEthernet. Devices 103, 105, 107, 109A-F and other devices (not shown)may be connected to one or more of the networks via twisted pair wires,coaxial cable, fiber optics, radio waves or other communication media.For example, the above connections can be made via the internet, bluetooth, WiFi, infrared, or any other known method of wirelesstransmission.

As shown in FIG. 1, devices 109A-F may include personal computers suchas desktops, laptops, notebooks, mobile telephones or smartphones withapplications and other functionality, a handheld device with Wi-Fi orother wireless connectivity (e.g., wireless enabled tablets, tabletcomputers, PDAs, and the like), displays with built-in or externalmemories and processors, or any other known computer, computing device,or handheld computer can also be connected to one or more of thenetworks described herein. It is also contemplated that other types ofdevices such as ATMs, kiosks, and other devices can be connected to oneor more of the networks described herein. These devices can be enabledto communicate with wireless access points which in one example can be aseries of cellular towers hosted by a service provider. Additionally,the wireless access points may be Wi-Fi (e.g., compatible with IEEE802.11a/b/g/and the like wireless communication standards) connectionsand the computing devices may obtain access to the Internet at theseconnections. Other known techniques may be used to allow devices toconnect with a network.

The term “network” as used herein and depicted in the drawings refersnot only to systems in which remote storage devices are coupled togethervia one or more communication paths, but also to stand-alone devicesthat may be coupled, from time to time, to such systems that havestorage capability. Consequently, the term “network” includes not only a“physical network” but also a “content network,” which is comprised ofthe data—attributable to a single entity—which resides across allphysical networks.

The components may include data server 103, web server 105, and clientcomputers 107, and devices 109 a-f. Data server 103 provides overallaccess, control and administration of databases and control software forperforming one or more illustrative aspects as described herein. Dataserver 103 may be connected to web server 105 through which usersinteract with and obtain data as requested. Alternatively, data server103 may act as a web server itself and be directly connected to theInternet. Data server 103 may be connected to web server 105 through thenetwork 101 (e.g., the Internet), via direct or indirect connection, orvia some other network. Users may interact with the data server 103using remote computers 107, devices 109 a-f, e.g., using a web browserto connect to the data server 103 via one or more externally exposed websites hosted by web server 105. Client computers 107, 109 may be used inconcert with data server 103 to access data stored therein, or may beused for other purposes. For example, from client device 107 or devices109 a-f a user may access web server 105 using an Internet browser, asis known in the art, or by executing a software application or app thatcommunicates with web server 105 and/or data server 103 over a computernetwork (such as the Internet).

Servers and applications may be combined on the same physical machines,and retain separate virtual or logical addresses, or may reside onseparate physical machines. FIG. 1 illustrates just one example of anetwork architecture that may be used, and those of skill in the artwill appreciate that the specific network architecture and dataprocessing devices used may vary, and are secondary to the functionalitythat they provide, as further described herein. For example, servicesprovided by web server 105 and data server 103 may be combined on asingle server.

Each component 103, 105, 107, 109 may be any type of known computer,server, or data processing device as discussed herein. Data server 103,e.g., may include a processor 111 controlling overall operation of therate server 103. Data server 103 may further include RAM 113, ROM 115,network interface 117, input/output interfaces 119 (e.g., keyboard,mouse, display, printer, or the like.), and memory 121. I/O 119 mayinclude a variety of interface units and drives for reading, writing,displaying, and/or printing data or files. Memory 121 may further storeoperating system software 123 for controlling overall operation of thedata server 103, control logic/data server software 125 for instructingdata server 103 to perform aspects as described herein, and otherapplication software 127 providing secondary, support, and/or otherfunctionality which may or may not be used in conjunction with one ormore aspects described herein. The control logic may also be referred toherein as the data server software 125. Functionality of the data serversoftware may refer to operations or decisions made automatically basedon rules coded into the control logic, made manually by a user providinginput into the system, and/or a combination of automatic processingbased on user input (e.g., queries, data updates, or the like).

Memory 121 may also store data used in performance of one or moreaspects, including a first database 129 and a second database 131. Insome embodiments, the first database may include the second database(e.g., as a separate table, report, or the like). That is, theinformation can be stored in a single database, or separated intodifferent logical, virtual, or physical databases, depending on systemdesign. Devices 105, 107, 109A-F may have similar or differentarchitecture as described with respect to device 103. Those of skill inthe art will appreciate that the functionality of data processing device103 (or device 105, 107, 109A-F) as described herein may be spreadacross multiple data processing devices, for example, to distributeprocessing load across multiple computers, to segregate transactionsbased on geographic location, user access level, quality of service(QoS), or the like.

One or more aspects may be embodied in computer-usable or readable dataand/or computer-executable instructions, such as in one or more programmodules, executed by one or more computers or other devices as describedherein. Generally, program modules include routines, programs, objects,components, data structures, or the like that perform particular tasksor implement particular abstract data types when executed by a processorin a computer or other device. The modules may be written in a sourcecode programming language that is subsequently compiled for execution,or may be written in a scripting language such as (but not limited to)HTML or XML. The computer executable instructions may be stored on acomputer readable medium such as a hard disk, optical disk, removablestorage media, solid state memory, RAM, or the like. As will beappreciated by one of skill in the art, the functionality of the programmodules may be combined or distributed as desired in variousembodiments. In addition, the functionality may be embodied in whole orin part in firmware or hardware equivalents such as integrated circuits,field programmable gate arrays (FPGA), and the like. Particular datastructures may be used to more effectively implement one or moreaspects, and such data structures are contemplated within the scope ofcomputer executable instructions and computer-usable data describedherein.

A schematic of an exemplary system is shown in relation to FIG. 2. Asshown in FIG. 2, the system may include one or more of a source system152, a message processor module 154, a database 156, a check systemmonitoring facility (CSMF) module 160, and a business activitymonitoring (BAM) module 158. The source system 152 can be configured torequest that source files be delivered to the database 156 through themessage processor module 154. The message processor module 154 can beconfigured to apply sorting rules and to apply validation rules to thedata. The sorting rules can order the data by database table clusteringorder for example, by listing the date and then the account number. Thevalidation rules may include checking the data for redundancies,inconsistencies or format issues associated with the data or, moreparticularly, checking to see if dates are dates, numeric data isnumeric, and the like.

The message processor module 154 can also be configured to transform thedata files into an XML format and to deliver XML formatted messages.Once the data is transferred to the database 156, the message processormodule 154 can also acknowledge the loading status of the files toeither the check system monitoring facility (CSMF) module 160 thebusiness activity monitoring unit (BAM) module 158, for example. Thedatabase 156 can be configured to store the source file data. In thisway, the delivery process can be monitored by either the CSMF module 160or the BAM module 158. The database 156 can also include an indexretrieval interface, native stored procedures, and a check indexarchive. These can provide various functions such as retrieval andinsert interfaces, along with a response function once the data isloaded. In one example, the database 156 can be configured to send aresponse to the message processor module 154 so the message processormodule 154 can acknowledge that the database 156 has received the sourcefile data.

An exemplary process for loading of data files is depicted in relationto FIG. 3. As shown in FIG. 3, the process can be initiated by a sourcesystem at step 202 in sending a source data file to be stored into adatabase to a landing zone at step 204. At step 204, the landing zonestep can be in a computing device in a network, server or cloud, forexample. The computing device can then register the file at step 206 andcreate control table records at step 208. A software component or modulecan extract the data from a source data file at the computing device atstep 210, and validate and transform the data from the data source fileat the computing device in step 212. The software component can alsoinclude a poller that watches for files from the source system. In analternative example, a quartz scheduler could be used to find files thatarrive.

In the validation step 212 the software component can check for certainerrors in the data by using a rule engine, for example, by checking thevarious data fields. In one example, the validation process can includereviewing the data for redundancies, inconsistencies or format issuesassociated with the data, such as, checking to see if dates are dates,numeric data is numeric, and the like. Also along with validation instep 212, the data can be transformed into a readable format for thedatabase, such as XML, text, image, zipped data, SQL or another computerreusable format for storage. The data may also be split into units ofwork (e.g., 2500 records per job). Additionally or alternatively, thedata may be divided (e.g., split up) into, for example, up to 5000records per load chunk (e.g., up to 5000 rows of XML data at a time),resulting in data loading that is consistent, free of formatting errors,and efficient. This efficient loading may be performed via, for example,distributed data facility (DDF) threads, as will be described in furtherdetail below.

Once the data is validated, transformed and split up accordingly, atstep 214, the software component can deliver the data to one or moremessage processor modules at either step 216 or 218. In this process,the software component can assign the data using round-robin schedulingor can determine which message processor module to send the data tobased on message processor bandwidth, or other techniques. Also in thisprocess the software component can effectively request that the data betransferred to a certain database at the message processor modules atsteps 216 or 218, which may depend on frequency, storage capacity,and/or speed to meet a service level agreement (SLA).

One or more connections or threads (e.g., DDF threads) may be created orotherwise generated between one or more message processor modules, suchas the message processor module B, and one or more mainframe database,such as the database B 226. In some aspects, DDF threads might not becreated for transferring data between a processor module and a databaseif the data will not be transferred using DDF threads and/or will betransferred using other technologies. For example, if DDF threads arecreated for transferring data between the message processor module B andthe database B 226, DDF threads might not be created for transferringdata between the message processor module A and database A 224.

At steps 216, 218, the message processor modules may send the data toone or more listener modules. For example, at step 216, the messageprocessor module(s) may send the data to a listener module, such asmodule A, via a message queue (MQ) based solution that might allowmessages to stored and processed slower. For example, module A may sendthe data as one or more messages to a message queue at any point intime. The message may be temporarily stored at the message queue. Thedatabase A may retrieve or otherwise access (e.g., request) the datastored at the message queue by module A at any point in time.

At step 218, the message processor module(s) may send the data viaDistributed Data Facility (DDF) connections or threads at a very highrate of speed for priority SLA. DDF may use connections from a mid-rangeserver to a mainframe database (via a native stored procedure) and maybe used by an application to access one or more databases. An exemplaryDDF connection or thread is a database access connection or thread. DDFconnections may be used to deliver large chunks of data, such as XMLdata, to databases, such as relational database management system(RDBMS) because multiple connections or threads (e.g., thousands) may becreated and used to simultaneously (e.g., in parallel) send data to orreceive data from the databases. Once the data hits the DDF thread, thedata may be perfected and be ready to be sent to the table and/ordatabases, such as mainframe databases. The perfected data may beverified and/or validated before the data is sent to the databases viaDDF connections. Use of DDF threads may significantly decrease themillion instructions per second (MIPS) cost or charges of loading datainto databases, such as by up to 25% (e.g., if 25% of the processingneeded to load the data goes away).

At step 220, 222, the listener modules may execute a native storedprocedure which will parse the XML and load the unit of work into aglobal temporary table, which can then be inserted or delivered intoeither database A at step 224 or database B at step 226. Since the datawas previously validated, the data can easily be added onto the selecteddatabase. Once the data is successfully transferred, the messageprocessor modules can post a response at step 228, 230, which canindicate that the data has been transferred to either database A ordatabase B at the message processor modules to acknowledge receipt ofthe data in the database at the computing device. As depicted in FIG. 3,the response may originate from either listener module A or listenermodule B. In this way, the software component on the computing devicecan acknowledge the data delivery.

The software component can also update the control records at step 234based on the acknowledgment message. At step 236 the acknowledgementreceipt can then be routed back to the source system through a NDM filetransfer. The acknowledgement can also be received at check systemmonitoring facility (CSMF) module at step 238 or a business activitymonitoring (BAM) module at step 240.

FIG. 4 shows another example process that can be used for extraction,transformation, validation and delivery of files. As shown in FIG. 4, aninput file 304 can be received from source systems 302 at a landing zone306, which can be a computing device in a network, server or cloud, forexample. The file can then undergo a data capture process 308 where thefile is stored into data tables 310. First, the poller 312 can identifyexistence of a new file and can apply a trigger 316 to trigger thescheduler 314 for processing file. A spring batch component 318 can thenuse modular rules 320 to split, sort, validate, clean, and transmit todatabase component as defined for each file type. The message queues 322can be used as the transport protocols which move the data from the datacapture platform to the database. Logging services 324 a, 324 b can becalled throughout the ingestion and loading process. Modular rules 326can be stored and reused to define new spring batch flows. The springJDBC 328 can be configured as a database communication layer for eventlogging and processing state details.

The example systems and processes discussed herein may help to reducedependency on a mainframe server, by conducting some of the validationprior to loading the data into the database. This may also result incost savings, since less expensive processors can be used. The processcan also be more flexible in reducing the need for having batch files.Also the use of DDF threads or connections can be reduced such that DDFthread use does not compete with online retrieval, which saves onbandwidth that can be used for other processes such as inserting andrequesting data from the database.

The processes disclosed herein may help to eliminate a significantnumber of processors and may be run on lower cost processors that do notrequire the payment of fees. The system can also help to eliminateredundant data extraction and transformation processes at dual databasesites because the validation and transformation can be done up frontbefore arriving at the database sites. This can help reduce the amountof processing required since these steps will only need to be performedonce. The system can be configured to reject any incoming file withinvalid data from the source system.

In one example, the system can help reduce the overall dependency on themainframe by validating the data prior to loading the data into thedatabase. Also the use of DDF threads can reduced such that the currentprocess does not compete with online retrieval. This can help certaindata insert activity more efficient as data is constantly loaded intothe database and data is constantly requested from the database.

In one example, a computer-implemented method for loading a source datafile can include one or more of the following steps: extracting datafrom the source data file at a computing device, validating the datafrom the source data file at the computing device, transforming the datafrom the source data file at the computing device, requesting the datato be transferred to a first database at a first message processormodule, delivering the data to the first database, or responding thatthe data has been transferred to the first database at the first messageprocessor module to acknowledge receipt of the data in the database atthe computing device. The computing device can be a server in oneexample. A first listener module can receive the data from the firstmessage processor and can load the data onto the first database. Themethod may also include one or more of: routing the acknowledgementreceipt to a source system, receiving the acknowledgement receipt at acheck system monitoring facility, receiving the acknowledgment receiptat a business activity monitoring unit, or updating control recordsbased on the acknowledgment receipt.

Additionally, a second message processor module can be configured todeliver the data to a second database. The second listener module canreceive the data from the second message processor and can load the dataonto the second database.

In another example method for loading a data file may include one ormore of the following steps: send a data file request from a sourcesystem, receive the request, apply validation rules, and transform thedata at a message processor module, deliver the data by the messageprocessor module to a database, respond to the message processor modulethat the data has been delivered, and acknowledge that the data has beendelivered by the message processor. A second message processor modulecan be configured to deliver the data to a second database. Theapparatus can also be configured to update control records based on theacknowledgment receipt, route the acknowledgement receipt to a sourcesystem module, receive the acknowledgement receipt at a check systemmonitoring facility module, or receive the acknowledgment receipt at abusiness activity monitoring module.

In another example method for loading a data file may include one ormore of the following steps: send a request from a source system moduleto deliver a source data file to a first database, extract data from thesource data file at a computing device, validate the data from the datasource file at the computing device, transform the data from the datasource file at the computing device, request the data to be transferredto a first database at a first message processor module, deliver thedata to the first database, and respond that the data has beentransferred to the first database at the first message processor moduleto acknowledge receipt of the data in the database at the computingdevice. The computing device may be server, and a second messageprocessor module can be configured to deliver the data to a seconddatabase. The method may also include updating control records based onthe acknowledgment receipt, routing the acknowledgement receipt to asource system. The load process can be monitored by a check systemmonitoring facility module, and the load process can also be monitoredby a business activity monitoring module.

In another example, an apparatus comprising: a processor; and a memoryfor storing computer readable instructions that, when executed by theprocessor, can cause the apparatus to perform the methods discussedherein. In another example, one or more non-transitory computer-readablemedia may have instructions stored thereon that, when executed, cause atleast one computing device to perform one or more aspects of the methodsdiscussed herein.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described above.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

What is claimed is:
 1. A method comprising: receiving, at one or moremid-range servers, one or more data files; extracting, by the one ormore mid-range servers, data from the one or more data files; generatinga plurality of distributed data facility (DDF) threads between the oneor more mid-range servers and a first database; sending, by the one ormore mid-range servers, a first portion of the extracted data to thefirst database using the generated plurality of DDF threads; and sendinga second portion of the extracted data to a message queue accessible toa second database.
 2. The method of claim 1, further comprising:determining whether the extracted data has an error, wherein sending thefirst portion of the extracted data to the first database and sendingthe second portion of the extracted data to the message queue areperformed in response to determining that the extracted data does nothave the error.
 3. The method of claim 1, wherein the first portion ofthe extracted data comprises a first data chunk and the second portionof the extracted data comprises a second data chunk, the method furthercomprising: dividing, by the one or more mid-range servers, theextracted data into the first data chunk and the second data chunk,wherein sending the first portion of the extracted data comprisessending the first data chunk, and wherein sending the second portion ofthe extracted data comprises sending the second data chunk.
 4. Themethod of claim 1, further comprising: receiving, from the seconddatabase, a request for the second portion of the extracted data; and inresponse to receiving the request, sending, via the message queue, thesecond portion of the extracted data to the second database.
 5. Themethod of claim 1, further comprising: transforming the extracted datainto XML data, wherein sending the first portion of the extracted datacomprises sending a first portion of the XML data, and wherein sendingthe second portion of the extracted data comprises sending a secondportion of the XML data.
 6. The method of claim 1, wherein sending thefirst portion of the extracted data to the first database using thegenerated plurality of DDF threads comprises simultaneously sending aplurality of data from the first portion of the extracted data acrossdifferent DDF threads of the plurality of DDF threads.
 7. The method ofclaim 1, further comprising: prior to sending the first portion of theextracted data to the first database, verifying and validating the firstportion of the extracted data.
 8. One or more server comprising: aprocessor; and memory storing computer-executable instructions that,when executed by the processor, cause the one or more server to: receiveone or more data files; extract data from the one or more data files;generate a plurality of distributed data facility (DDF) threads betweenthe one or more server and a first database; send a first portion of theextracted data to the first database using the generated plurality ofDDF threads; and send a second portion of the extracted data to amessage queue accessible to a second database.
 9. The one or more serverof claim 8, wherein the memory stores computer-executable instructionsthat, when executed by the processor, cause the one or more server to:determine whether the extracted data has an error, wherein sending thefirst portion of the extracted data to the first database and sendingthe second portion of the extracted data to the message queue areperformed in response to determining that the extracted data does nothave the error.
 10. The one or more server of claim 8, wherein the firstportion of the extracted data comprises a first data chunk and thesecond portion of the extracted data comprises a second data chunk, andwherein the memory stores computer-executable instructions that, whenexecuted by the processor, cause the one or more server to: divide theextracted data into the first data chunk and the second data chunk,wherein sending the first portion of the extracted data comprisessending the first data chunk, and wherein sending the second portion ofthe extracted data comprises sending the second data chunk.
 11. The oneor more server of claim 8, wherein the memory stores computer-executableinstructions that, when executed by the processor, cause the one or moreserver to: receive, from the second database, a request for the secondportion of the extracted data; and in response to receiving the request,send, via the message queue, the second portion of the extracted data tothe second database.
 12. The one or more server of claim 8, wherein thememory stores computer-executable instructions that, when executed bythe processor, cause the one or more server to: transform the extracteddata into XML data, wherein sending the first portion of the extracteddata comprises sending a first portion of the XML data, and whereinsending the second portion of the extracted data comprises sending asecond portion of the XML data.
 13. The one or more server of claim 8,wherein sending the first portion of the extracted data to the firstdatabase using the generated plurality of DDF threads comprisessimultaneously sending a plurality of data from the first portion of theextracted data across different DDF threads of the plurality of DDFthreads.
 14. The one or more server of claim 8, wherein the memorystores computer-executable instructions that, when executed by theprocessor, cause the one or more server to: prior to sending the firstportion of the extracted data to the first database, verify and validatethe first portion of the extracted data.
 15. A non-transitorycomputer-readable media storing computer-readable instructions that,when executed by one or more server, cause the one or more server to:receive one or more data files; extract data from the one or more datafiles; generate a plurality of distributed data facility (DDF) threadsbetween the one or more server and a first database; send a firstportion of the extracted data to the first database using the generatedplurality of DDF threads; and send a second portion of the extracteddata to a message queue accessible to a second database.
 16. Thenon-transitory computer-readable media of claim 15, storingcomputer-readable instructions that, when executed by the one or moreserver, cause the one or more server to: determine whether the extracteddata has an error, wherein sending the first portion of the extracteddata to the first database and sending the second portion of theextracted data to the message queue are performed in response todetermining that the extracted data does not have the error.
 17. Thenon-transitory computer-readable media of claim 15, wherein the firstportion of the extracted data comprises a first data chunk and thesecond portion of the extracted data comprises a second data chunk, andwherein the non-transitory computer-readable media storescomputer-readable instructions that, when executed by the one or moreserver, cause the one or more server to: divide the extracted data intothe first data chunk and the second data chunk, wherein sending thefirst portion of the extracted data comprises sending the first datachunk, and wherein sending the second portion of the extracted datacomprises sending the second data chunk.
 18. The non-transitorycomputer-readable media of claim 15, storing computer-readableinstructions that, when executed by the one or more server, cause theone or more server to: receive, from the second database, a request forthe second portion of the extracted data; and in response to receivingthe request, send, via the message queue, the second portion of theextracted data to the second database.
 19. The non-transitorycomputer-readable media of claim 15, storing computer-readableinstructions that, when executed by the one or more server, cause theone or more server to: transform the extracted data into XML data,wherein sending the first portion of the extracted data comprisessending a first portion of the XML data, and wherein sending the secondportion of the extracted data comprises sending a second portion of theXML data.
 20. The non-transitory computer-readable media of claim 15,wherein sending the first portion of the extracted data to the firstdatabase using the generated plurality of DDF threads comprisessimultaneously sending a plurality of data from the first portion of theextracted data across different DDF threads of the plurality of DDFthreads.